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Abstract 

We exhibit the rich structure of the set of correlated equilibria by analyzing the 
simplest of polynomial games: the mixed extension of matching pennies. We show 
that while the correlated equilibrium set is convex and compact, the structure of its 
extreme points can be quite complicated. In finite games the ratio of extreme correlated 
to extreme Nash equilibria can be greater than exponential in the size of the strategy 
spaces. In polynomial games there can exist extreme correlated equilibria which are 
not finitely supported; we construct a large family of examples using techniques from 
ergodic theory. We show that in general the set of correlated equilibrium distributions 
of a polynomial game cannot be described by conditions on finitely many moments 
(means, covariances, etc.), in marked contrast to the set of Nash equilibria which is 
always expressible in terms of finitely many moments. 



1 Introduction 

Correlated equilibria are a natural generalization of Nash equilibria introduced by Aumann 
[]]. They are defined to be joint probability distributions over the players' strategy spaces, 
such that if each player receives a private recommendation sampled according to the dis- 
tribution, no player has an incentive to deviate unilaterally from playing his recommended 
strategy. In finite games the set of correlated equilibria is a compact convex polytope, and 
therefore seemingly much simpler than the set of Nash equilibria, which can be essentially 
any algebraic variety jl]. Even in the simple case of two-player finite games, the set of Nash 
equilibria is a union of finitely many polytopes: seemingly more complicated than the set of 
correlated equilibria. 
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Nonetheless we will see that there are two-player zero-sum games in which the set of 
correlated equilibria has many more extreme points than the set of Nash equilibria has. 
This behavior does not seem to be pathological in any way: it occurs in very simple finite 
games and the simplest of infinite games. We take this as evidence that this complexity is 
likely to be quite common. 



Contributions 

• We give a family of examples of two-player zero-sum finite games in which the set of 
Nash equilibria has polynomially many extreme points (Section [3b , while the set of 
correlated equilibria has factorially many extreme points (Section Mh. 

For bimatrix games, this shows that while extreme Nash equilibria are a subset of 
the extreme correlated equilibria (see Related Literature below), enumerating all the 
extreme correlated equilibria is in general a bad way of computing all the extreme Nash 
equilibria. In particular, it would be faster to enumerate all subsets of the strategy 
spaces (there are "only" exponentially many) and check whether each was the support 
of a Nash equilibrium. 

• We give a related example of a continuous game with strategy sets equal to [—1, 1] 
and bilinear utility functions. This game is just the mixed extension of matching 
pennies, but we show that it has extreme correlated equilibria with arbitrarily large 



finite support (Proposition 4.5) and also with infinite support (Proposition 4.12). This 
is in contrast to the extreme Nash equilibria, which always have uniformly bounded 
finite support in zero-sum games with polynomial utilities [T2] . 

Once the existence of Nash equilibria in continuous games has been established [TO] , 
it is straightforward to show that polynomial game^j] admit Nash equilibria with finite 
support [121 [21] • There are more elementary ways of showing that continuous games 
have correlated equilibria [UJ, but to the authors' knowledge there is no proof that 
polynomial games have finitely supported correlated equilibria which does not rely on 
the existence of Nash equilibria. This example shows that the plausible-sounding proof 
idea that all extreme correlated equilibria are finitely supported simply isn't true. 



Comparing Proposition 4.14 with this example shows that in general there is no finite- 
dimensional description of the set of correlated equilibria of a zero-sum polynomial 
game. That is to say, one cannot check if a measure is a correlated equilibrium merely 
by examining finitely many generalized moments (parameters such as mean, covariance, 
etc. - any compactly supported distribution can be specified by countably many such 
parameters). Such a description for the Nash equilibria has been known for over fifty 
years [12] . 



^or the purposes of this paper a polynomial game is one in which the strategy spaces are compact 
intervals and the utility functions are polynomials in the players' strategies. 
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Intuitively, the reason for this difference is that being a correlated equilibrium is a 
statement about conditional distributions, and these are too delicate to be controlled 
by finitely many moments. This example confirms the intuition. 

Experience from finite games suggests that correlated equilibria should be easier to 
compute than Nash equilibria. While there are computational methods which con- 
verge asymptotically to correlated equilibria of polynomial games [22], the only exact 
algorithm the authors are aware of consists of computing a Nash equilibrium by quan- 
tifier elimination (extremely slow), which is possible because of the finite-dimensional 
description. In particular, no provably efficient method for computing correlated equi- 
libria of polynomial games exactly or approximately is known. The lack of a finite- 
dimensional description of the problem seems to be an important part of what makes 
it difficult. 

Related Literature The geometry of Nash and correlated equilibria has been studied 
extensively. Therefore we only mention work below if it is directly connected to ours and we 
do not attempt to be exhaustive. 

The result most closely related to the present paper states that in two-player finite 
games, extreme Nash equilibria (viewed as product distributions) are a subset of the extreme 
correlated equilibria. Cripps [3] and Evangelista and Raghavan [7J proved this independently. 
This result shows that it makes sense to compare the number of extreme Nash and correlated 
equilibria. It also raises the natural question of whether all extreme Nash equilibria could be 
enumerated efficiently by enumerating the extreme correlated equilibria. We show that there 
can be many more extreme correlated equilibria than extreme Nash equilibria, answering this 
question in the negative. 

In a similar vein, Nau et al. [17] show that for non-trivial finite games with any number of 
players, the Nash equilibria lie on the boundary of the correlated equilibrium polytope. With 
three or more players, the Nash equilibria need not be extreme correlated equilibria. For 
example consider the three-player poker game analyzed by Nash in [16J which has rational 
payoffs, hence rational extreme correlated equilibria, but whose unique Nash equilibrium 
uses irrational probabilities. 

Separable games, a generalization of polynomial games, were first studied around the 
1950's by Dresher, Karlin, and Shapley in papers such as [6], [5], and [IB], which were later 
combined in Karlin's book [12]. Their work focuses on the zero-sum case, which contains 
some of the key ideas for the nonzero-sum case. In particular, they show how to replace the 
infinite-dimensional mixed strategy spaces (sets of probability distributions over compact 
metric spaces) with finite-dimensional moment spaces. Caratheodory's theorem [2] then 
applies to show that finitely-supported Nash equilibria exist. 

There are many similarities between separable games and finite games whose payoff matri- 
ces satisfy low-rank conditions. Lipton et al. [H] consider two-player finite games and provide 
bounds on the cardinality of the support of extreme Nash equilibrium strategies in terms 
of the ranks of the payoff matrices. The main technical tool here is again Caratheodory's 
theorem. 
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Germano and Lugosi show that in finite games with three or more players there exist 
correlated equilibria with smaller support than one might expect for Nash equilibria [9]. The 
proof is geometrical; it essentially views correlated equilibria as living in a subspace of low 
codimension and it too uses Caratheodory's theorem [2]. 

The bounds on the support of equilibria in finite and separable games of the previous 
three paragraphs are all synthesized in [20]; the portion on Nash equilibria has appeared in 
[2"T] . The general idea is that simple payoffs (low-rank matrices, low-degree polynomials, etc.) 
lead to simple Nash equilibria (small support), and those in turn lead to simple correlated 
equilibria (small support again). 

To produce upper bounds on the minimal support of correlated equilibria which depend 
only on the rank of the payoff matrices and not on the size of the strategy sets, this work 
does not bound the support of all extreme correlated equilibria, but rather only those whose 
support is contained inside a Nash equilibrium of small support, which must exist. Similar 
results hold for polynomial games with, for example, degree used in place of rank (the notions 
of degree and rank are generalized in [20] and [21J). 

This work left open the question of whether all extreme correlated equilibria have support 
size which can be bounded in terms of the rank of the payoff matrices, independently of the 
size of the strategy sets. Here we show that this is not the case, because our examples 
have payoffs which are of rank 1 and extreme correlated equilibria of arbitrarily large, even 
infinite, support. 

Correlated equilibria without finite support have been defined and studied by several 
authors. An important example of this line of research is the paper by Hart and Schmeidler 
[XT] . The definition of correlated equilibria presented in [11 J is convenient for proving some 
theoretical results (they focus on existence) but not usually for computation. 

The authors of the present paper have developed several equivalent characterizations of 
correlated equilibria in continuous games which are more suitable for computation [22]. One 
of these forms the basis for the analysis in Section [4] below. Other such characterizations 
lead to algorithms for approximating correlated equilibria of continuous games [22J. 

Outline The remainder of this paper is organized as follows. Section 2 introduces the 
examples to be studied. The two types of example are closely related - the finite game 
examples are just restrictions of the strategy spaces in the infinite game example to fixed 
finite sets. This allows us to analyze both examples on equal footing. In Section 3 we define 
and compute the extreme Nash equilibria of these examples, counting them in the finite 
game example. Then we define and analyze the extreme correlated equilibria in Section 4. 
This analysis is somewhat long and at times technical, so we present a detailed roadmap 
before beginning. We close with Section 5, where we outline directions for future work. 

2 Description of the examples 

First we fix notation. When S is a topological space, A(5) will denote the set of Borel 
probability measures on S and A*(S) the set of finite Borel measures on S. In particular 
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(u x ,u Y ) 


x = — 1 x = 1 


y = -\ 
y = l 


(1,-1) (-1,1) 
(-1,1) (1,-1) 



Table 1: Utilities for matching pennies 

A(S) is the set of measures in A*(S) with unit mass. If S is finite it will be given the discrete 
topology by default so A (5) is a simplex and A*(S) is an orthant in Rl s L We abuse notation 
and write the measure of a singleton {p} as fi(p) rather than For any p G S, define 

S p G A(5) to be the measure which assigns unit mass to the point p. Let / = [—1, 1] C K. 

We will focus on two related examples, one with finite strategy sets and one with infinite 
strategy sets. We will develop them in parallel by analyzing arbitrary games satisfying the 
following condition. The condition does not have any game theoretic content; it was merely 
chosen for simplicity and the results to which it leads. 

Assumption 2.1. The game is a zero-sum strategic form game with two players, called 
X and Y. The strategy sets Cx and Cy are compact subsets of I = [—1,1], each of 
which contains at least one positive element and at least one negative element. Player 
X chooses a strategy x G C x and player Y chooses y G Cy. The utility function^] are 

u x (x,y) = xy = -u Y (x,y). 

Example 2.2. Fix an integer n > 0. Let C x and Cy each have 2n elements, n of which are 
positive and n of which are negative. If we take n = 1 and Cx = Cy = { — 1,1} then we 
recover the matching pennies game, as shown in Table [TJ 

Example 2.3. Let C x = Cy = [—1, 1]. Then the game is essentially the mixed extension of 
matching pennies. That is to say, suppose two players play matching pennies and choose 
their strategies independently, playing 1 with probabilities p G [0,1] and q G [0,1]. Define 
the utilities for the mixed extension to be the expected utilities under this random choice 
of strategies. Letting x = 2p — 1 and y — 2q — 1, the utility to the first player is xy and 
the utility to the second player is — xy. Therefore this example is the mixed extension of 
matching pennies, up to an affine scaling of the strategies. 

Usually one looks at pure equilibria of the mixed extension of a game; these are exactly 
the mixed equilibria of the original game. We will instead be looking at mixed Nash equi- 
libria and correlated equilibria of the mixed extension itself, a game with a continuum of 
actions. The relationship between correlated equilibria of the mixed extension and those of 
the original game is much more complicated than the corresponding relationship for mixed 
Nash equilibria. This drives the results of the paper. 

2 By inspection of the utilities we can see that for any Cx and Cy with at least two points, the rank of 
this game in the sense of [21] is (1, 1) (and in fact also in the stronger sense of Theorem 3.3 of that paper). 
The notion of the rank of a game is related to the rank of the payoff matrices and will not play a significant 
role in this paper; we merely wish to note that under this definition of complexity of payoffs, the games we 
consider are extremely simple. 
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3 Extreme Nash equilibria 



We now characterize and count the extreme points of the sets of Nash equilibria in games 



satisfying Assumption C 1A_. Since the games are zero-sum, the set of Nash equilibria can be 
viewed as a Cartesian product of two (weak*) compact convex sets, the sets of maximin and 
minimax strategies [10J. The Krein-Milman theorem completely characterizes such sets by 
their extreme points [18], explaining our focus on extreme points throughout. 

We define Nash equilibria in two-player games, which will be sufficient for our purposes, 
as well as the standard notions of extreme point and extreme ray from convex analysis. 

Definition 3.1. A Nash equilibrium is a pair (a, r) G A(Cx) xA(Cy) such that ux(x, r) < 
ux(cr, t) for all x G Cx and My (a, y) < uy(a, r) for all y G Cy (where we extend utilities by 
expectation in the usual fashion ux(x,r) = J ux(x,y) dr(y), etc.). 

In other words, a Nash equilibrium is a strategy pair in which each player is playing a 
best reply to his opponent's strategy. 

Definition 3.2. A point x in a (usually convex) subset K of a real vector space is an 
extreme point if x = Xy + (1 — X)z for y, z G K and A G (0, 1) implies x = y = z. 

The related notion of extreme ray will not be used until the next section, but we record 
it here for comparison. 

Definition 3.3. A convex set K such that x G K and A > implies Ax G K is called a 
convex cone. A point x ^ is an extreme ray of the convex cone K if x — y + z and 
y, z G K implies that y or z is a scalar multiple of x. 



The Nash equilibria of games satisfying Assumption |2.1| take the following particularly 
simple form. 

Proposition 3.4. A pair (cr, r) G A(Cx) x A(Cy) is a Nash equilibrium of a game satisfying 



Assumption 2.1 if and only if J xdo~(x) = J ydr{y) = 0. 



Proof. If J xdcr(x) = then uy(o-,y) = for all y G Cy, so any r G A(CV) is a best 
response to a. If J y dr(y) — as well then a is also a best response to r, so (cr, r) is a Nash 
equilibrium. 

Suppose for a contradiction that there exists a Nash equilibrium (a, r) such that J x da(x) > 
0; the other cases are similar. Player y must play a best response, so J ydr(y) < 0, which 
is possible by assumption. Player x plays a best response to that, so J xda(x) < 0, a 
contradiction. □ 

We introduce the notion of extreme Nash equilibrium in two-player zero-sum games. For 
an extension of this definition to two-player finite games and a proof that extreme Nash 
equilibria are always extreme points of the set of correlated equilibria in this setting, see [3] 
or 0. 
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Definition 3.5. In a two-player zero-sum game, maximin and minimax strategies are 
those mixed strategies for player X and Y, respectively, which appear in a Nash equilibrium. 
A Nash equilibrium of a zero-sum game is called extreme if a and r are extreme points of 
the maximin and minimax sets, respectively. 



Applying Proposition 3.4 to this definition, we can characterize the extreme Nash equi- 



libria of games satisfying Assumption |2.1| 

Proposition 3.6. Consider a game satisfying Assumption 2.1 A pair (a, r) e A(Cx) 



A(Cy) is an extreme Nash equilibrium if and only if a and r are each either 5q or of the 
form aS u + (35 v where u < 0, v, a, {3 > ; a + {3 = 1, and au + j3v = 0. 



Proof. By Proposition |3.4| we must show that these distributions are the extreme points of 
the set of probability distributions having zero mean. Since 5q is an extreme point of the 
set of probability distributions, it must be an extreme point of the subset which has zero 
mean. To see that a5 u + (35 v is also an extreme point, suppose we could write it as a convex 
combination of two other probability distributions with zero mean. The condition that both 
be positive measures implies that both must be of the form a'5 u + (3'5 V . But a and (3 as 
specified above are the unique coefficients which make this be a probability measure with 
zero mean. Therefore a' = a and 0' = f3, so a5 u + /35 v cannot be written as a nontrivial 
convex combination of probability distributions with zero mean, i.e., it is an extreme point. 

Suppose cr were an extreme point which was not of one of these types. Then a could 
not be supported on one or two points, so either [0, 1] or [—1,0) could be partitioned into 
two sets of positive measure. We will only treat the first case; the second is similar. Let 
[0, 1] = A U B where A PI B = and c(A), a(B) > 0. Since a has zero mean we must have 
ct([-1,0)) > as well. 

For a set D we define the restriction measure a\o by o~\d(C) = o~(D R C) for all C. 
Then a = o\a + c|b + c|[-i,o)- Let a = j A xda(x),b = j B x da(x), and c = J, 10 ^xda(x). 
Since cr([— 1, 0)) > and x is less than zero everywhere on [—1, 0), we must have c < and 
similarly a, b > 0. By assumption a + b + c = 0. Therefore we can write: 

a \ / b 



O = \ (T\A + |^j-CT| [-1,0) I + y?\B + j^|Cr|[-l,0) 

Being an extreme point of the set of probability measures with zero mean, a must be an 
extreme ray of the set of positive measures with first moment equal to zero. But this means 
that we cannot write a = o~\ + cr 2 where the <7j are positive measures with zero first moment 
unless o~i is a multiple of a . Neither of the measures in parentheses above is a multiple of a, 
so we have a contradiction. □ 

We illustrate this proposition on both examples introduced in Section [2] 

Example 2.2 (cont'd). In this case neither Cx nor Cy contains zero, so the only extreme 
Nash equilibria are those in which a and r are of the form a5 u + (35 v for u < and v > 0. 
For any choice of u and v simple algebra gives unique a and (3 satisfying the conditions of 



Proposition 3.6 There are n possible choices for each of u and v for each of the two players, 



so there are n extreme Nash equilibria. 
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Example 2.3 (cont'd). Since Cx = Cy = [—1,1], there are infinitely many extreme Nash 
equilibria in this case. However, they are all finitely supported and the size of the support 
of each player's strategy is always either one or two. Furthermore the condition that (cr, r) 
be a Nash equilibrium is equivalent to both having zero mean. This illustrates the general 
facts that in games with polynomial utility functions the Nash equilibrium conditions only 
involve finitely many moments of a and r (in this case, only the mean) and the extreme Nash 
equilibria (when defined, say for zero-sum games) have uniformly bounded support [T2] . 



4 Extreme correlated equilibria 

In this section we will show that even in finite games, the number of extreme correlated 
equilibria can be much larger than the number of extreme Nash equilibria. It makes sense to 
compare these because all extreme Nash equilibria of a two-player game, viewed as product 
distributions, are automatically extreme correlated equilibria [31 [7]. 

In the case of polynomial games we will show that there can be extreme correlated 
equilibria with arbitrarily large finite support and with infinite support. This implies that the 
set of correlated equilibria cannot be characterized in terms of finitely many joint moments. 



Roadmap The analysis proceeds in several steps which will be technical at times, so we 
start with an outline of what follows. 



We begin by defining correlated equilibria in games satisfying Assumption 2.1 
characterization from 



using a 



Proposition 4.4 shows that this characterization can be simplified because of our choice 
of utility functions. 

We use this characterization to construct a family of finitely supported extreme corre- 



lated equilibria in Proposition 4.5 



Then we note that all extreme correlated equilibria of the games in Example 2^ are of 
this form, so this allows us to count the extreme correlated equilibria and determine 
their asymptotic rate of growth as the number of pure strategies grows. 

Next we introduce some ideas from ergodic theory. With these tools in hand, we 



construct in Proposition 4.12| a large family of extreme correlated equilibria without 



finite support for the game in Example 2.3 



Finally we show that if a set can be represented by finitely many moments then all 
its extreme points have uniformly bounded finite support. This shows that the set 



of correlated equilibria of the game in Example |2.3| cannot be represented by finitely 
many moments and completes the analysis. 
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Having completed the roadmap, we are ready to begin. Correlated equilibria are meant 
to capture the notion of a joint distribution of private recommendations to the two players 
such that neither player can expect to improve his payoff by deviating unilaterally from 
his recommendation. For finitely supported probability distributions and games satisfying 



Assumption 2.1, this can be written as per the standard definition (see [15] or [8]): 



Definition 4.1. A finitely supported probability distribution \x G A(Cx x Cy) is a corre- 



lated equilibrium of a game satisfying Assumption 2.1 if 



^2 M^^N - x 'y] ^ 

yec Y 

for all x, x' G Cx and 

x£C x 

for all y, y' G Cy (note that these sums are finite by the assumption on //). 

The standard definition extending this notion to arbitrary (not necessarily finitely sup- 
ported) distributions is given in [UJ. This definition is difficult to compute with, so we will 
use the following equivalent characterization. 

Proposition 4.2 Q22J). A probability distribution fi G A(Cx x Cy) is a correlated equilib- 



rium of a game satisfying Assumption 2.1 if and only if 



(xy — x'y) dfi(x, y) > and / (xy — xy') djj(x, y) < 

Axl JlxA 



for all x 1 G Cx, y' G Cy, and measurable A C I. 



Proof. When jj, is finitely supported this is clearly equivalent to Definition 4.1 The general 



case is part (1) of Corollary 2.14 in (22] with the present utilities substituted in. □ 

Note that these conditions are homogeneous (that is, invariant under positive scaling) in 
jj,. The only condition on fi that is not homogeneous is the probability measure condition 
fi(I x I) — 1. We will often ignore this condition to avoid having to normalize every expres- 
sion, referring to a measure fi G A*(Cx x Cy) satisfying the conditions of the proposition as 
a correlated equilibrium. 

Definition 4.3. When we need to distinguish these notions, we will refer to a measure 



ji G A*(Cx x Cy) satisfying the conditions of Proposition 4.2 as a homogeneous corre- 
lated equilibrium and a measure \x G A(Cx x Cy) satisfying the conditions as a proper 
correlated equilibrium. In the context of homogeneous correlated equilibria the term 
extreme will refer to extreme rays; for proper correlated equilibria it will refer to extreme 
points. 
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When u ^ is a homogenous correlated equilibrium, .} T , ix is a proper correlated 
equilibrium. The set of homogenous correlated equilibria is a convex cone. The extreme 
rays of this cone are exactly those measures which are positive multiples of the extreme 
points of the set of proper correlated equilibria. 

The following proposition characterizes correlated equilibria of games satisfying Assump- 



tion 2.1 and is analogous to Proposition 3.4 for Nash equilibria. Note how the Nash equi- 
librium measures were characterized in terms of their moments but the correlated equilibria 
are not. Whereas the Nash equilibria are pairs of mixed strategies with zero mean for each 
player, condition (J3]) of this proposition says that the correlated equilibria are joint distri- 
butions such that regardless of each player's own recommendation, the conditional mean of 
his opponent's recommended strategy is zero. 



Proposition 4.4. For a game satisfying Assumption 2.1 and a measure fi G A*(Cx X Cy) 
such that xy ^ ji-a.e., the following are equivalent: 

1. fx is a correlated equilibrium; 

2. 

k x (A) := / xydLi(x,y) and K y (A) :— / xydfx(x,y) 

JAxI JlxA 

are both the zero measure, i.e., equal zero for all measurable A C I; 

3. 

X X (A) := / ydn(x,y) and X y (A) := / xdfx(x,y) 

JAxI JlxA 

are both the zero measure. 



4.2 



Proof. ([T] =>■ [2]) We will consider only k x \ k v is similar. The conditions of Proposition 
with A — I imply that 

x \ ydLi(x,y)< / xy d{i(x,y) < y' xdfi(x,y) 

Jlxl Jlxl Jlxl 

for all x' G Cx,y' G Cy- By assumption it is possible to choose x' and y' either pos- 
itive or negative, so f IxI xyd[i(x,y) = 0. A similar argument with any A implies that 
J, jXy d/x(x,y) > 0. Therefore we have 



xydfx(x,y)— / xyd/x(x,y)+ / xydLi(x,y) > + = 

Ixl JAxI J(I\A)xI 

for all A, so the inequality must be tight and we get f AI xy dfx(x, y) — for all A. 

([2] <^> [3]) By definition dn x = x d\ x and by assumption X x (0) = 0. If one of these measures 
is zero then so is the other, and respectively with y in place of x. 

([2] & [3] =>■ [l]) The integrals in Proposition 4.2 vanish. □ 
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Proposition 4.5. Fix a game satisfying Assumption 2.1, Let k > be even and x\, . . . ,X2k 
and |/i , ... , jj2k be such that: 

1. Xi G Cx and y^ G Cy are all nonzero; 

2. the sequence xi,x%, . . . ,X2k-i has distinct elements and alternates in sign; 

3. the sequence yi,ys, ■ ■ ■ ,V2k-i has distinct elements and alternates in sign; 

4- x 2i = x 2 i-i and y 2 i = y 2 i+\ f or oil i when subscripts are interpreted mod 2k. 
Then /x = Y^h=i J^l^i^im) ^ s an ex ^ reme correlated equilibrium. 

Proof. To show that /i is a correlated equilibrium define dn(x,y) = xy d a[x, y). Then k 

^2k 



2~2i=i s ^S n ( x i) sign (j/i ) <5(^^ >yi ) . Defining the projection k x as in Proposition 

2k k 



4.4 



we have 



Kx = X] si S n ( x *) Mgn^)^ = ^ sign(x 2i ) (sign(y 2i ) + sign(y 2 i-i)) S X2i 
i=i i=i 
k 

= ^sign(x 2 i)(0)5 X2i = 



i=i 



because x 2 i = x 2 i-i and y 2 i differs in sign from y 2 i-\ by assumption. The same argument 
shows that n y = 0, so /x is a correlated equilibrium. 

To see that /x is extreme, suppose /x = /x' + /x" where // and /x" are correlated equi- 
libria. Clearly /x' = YjL=i a 'i°~{xi,yi) f° r some a « > 0. Define <zV = xy d/jf(x,y), so «' = 
Y?iLi a i x iyi$(x i>yi )- By assumption 

i=l 

is the zero measure. Since the x 2i are distinct and nonzero we must have a 2i _iy 2i _i + a 2i y 2i = 
for all i. Similarly since n' y = we have a 2 i + \x 2 i + i + a 2 iX 2 i = for all i (with subscripts 
interpreted mod 2k). 

The Xi and yi are all nonzero, so fixing one fixes all the others by these equations. 
That is to say, these equations have a unique solution up to multiplication by a scalar, so 
fjf is a positive scalar multiple of /x. But the splitting /x = /x' + /x" was arbitrary, so /x is 
extreme. □ 



An argument along the lines of the proof of Proposition |4.5| shows that any finitely 
supported correlated equilibrium /x whose support does not contain any points with x = or 
y — can be written as /x = /x' + /x" where /x' 7^ is a correlated equilibrium and /x" 7^ is a 
correlated equilibrium of the form studied in Proposition 4J3 Therefore a finitely supported 
/x cannot be extreme unless it is of this form. 
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1 



0.5 ■ 



-0.5 ■ 



-0.5 



0.5 



Figure 1: The support of an extreme correlated equilibrium. In the notation of Proposi- 



tion 4.5, k = 2,xi = 0.4, x 3 = -0.6, y x = 0.2, and y 3 = -0.8. 



1 



0.5 ■ 



-0.5 



-0.5 



0.5 



Figure 2: The support of another extreme correlated equilibrium. In the notation of Propo- 
sition |45j k = 4, x x = 0.4, x 3 = -0.4, x 5 = 0.6, x 7 = -0.6, y x = 0.6, y 3 = -0.4, y 5 = 0.4, 
and y-j = —0.6. 
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Example 2.2 (cont'd). For some examples of the supports of extreme correlated equilibria of 
games of this type, see Figures [T] and |2j 

To count the number of extreme correlated equilibria of this game we must count the 



number of essentially different sequences of Xi and yi of the type mentioned in Proposition 4.5 



Fix k and let k = 2r where 1 < r < n. Note that cyclically shifting the sequences of x^s 
and Hi's by two does not change fi, nor does reversing the sequence. Therefore we can 
assume without loss of generality that x±,yi > 0. We then have n possible choices for 

Xi,yi,xs, and y 3 , n — 1 possible choices for £5, 27,7/5, and 2/7, etc., for a total of f 7^3^ 

possible choices of the Xi and yi. These will always be essentially different (i.e., give rise 
to different \x) unless we cyclically permute the sequences of Xi and yi by some multiple of 
four, in which case the resulting sequence is essentially the same. The number of such cyclic 
permutations is r. Therefore the total number of extreme correlated equilibria is 

i 



«M = E;((S^)i) 



We will see that e(n) = 9 (^(n!) 4 ). That is to say, e(n) is asymptotically upper and 
lower bounded by a constant times ^(n\) A . The expression ^(fi!) 4 is just the final term in 
the summation for e(n), so the lower bound is clear. Define 

Then f(n) > 1 for all n. We will now show that f(n) is also bounded above. Intuitively 
this is not surprising since the terms in the summation for f{n) die off extremely quickly as 
s grows. 

For all 1 < s < n — 1 we have that the ratio of term s + 1 in the summation to term s is: 

n-s-l ((s+1)!) 4 _ n — S ! 1 



n-s-1 (s + 1) 



so for n > 1 we can bound the sum by a geometric series: 

t(„\ 1 = y n 1 < n y 1 8n < 16 
JK ' ' (s\y ~ n- 1 ^8* 7(n - 1) ~ 7 ' 

Therefore 1 < f(n) < y for all n, so e(n) = G (^(^0 4 ) as claimed. Comparing this 
to the results of the previous section in which we saw that the number of extreme Nash 
equilibria of this game is n A , we see that in this case there is a super-exponential separation 
between the number of extreme Nash and the number of extreme correlated equilibria. This 
implies, for example, that computing all extreme correlated equilibria is not an efficient 
method for computing all extreme Nash equilibria, even though all extreme Nash equilibria 
are extreme correlated equilibria and recognizing whether an extreme correlated equilibrium 
is an extreme Nash equilibrium is easy. There are simply too many extreme correlated 
equilibria. 
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Next we will prove a more abstract version of Proposition 4.5 which includes certain 
extreme points which are not finitely supported. Before doing so we need a brief digression 
to ergodic theory. The first definition is the standard definition of compatibility between 
a measure and a transformation on a space. The second definition expresses one notion 
of what it means for a transformation to "mix up" a space - in this case that the space 
cannot be partitioned into two sets of positive measure which do not interact under the 
transformation. Then we state the main ergodic theorem and a corollary which we will 



apply to exhibit extreme correlated equilibria of games satisfying Assumption 2.1 



Definition 4.6. Given a measure \i G A*(S) on a space S, a measurable function g : S — )■ S 
is called (/i-)measure preserving if /z((7 _1 (A)) = fJ,{A) for all measurable A C S. Note that 
if g is invertible (in the measure theoretic sense that an almost everywhere inverse exists), 
then this is equivalent to the condition that fi(g(A)) = fx(A) for all A. 

Definition 4.7. Given a measure /x G A*(S), a ^-measure preserving transformation g is 
called ergodic if fj,(A A g~ 1 {A)) = implies n{A) = or fi(A) = fi(S), where AAB denotes 
the symmetric difference (A \ B) U {B \ A). 

Example 4.8. Fix a finite set S and a function g : S — > S. Let /i be counting measure on 
S. Then g is measure preserving if and only if it is a permutation. In this case a set T 
satisfies /i(^ _1 (T) A T) = if and only if (yf~ 1 (T) = T if and only if T is a union of cycles of 
g. Therefore g is ergodic if and only if it consists of a single cycle. 

Example 4.9. Fix a G R. Let S = [0,1) and let \i be Lebesgue measure on S. Define 
g : S — > S by g(x) = (x + a) mod 1 = (x + a) — \_x + a\ . Then g is /i-measure preserving 
because Lebesgue measure is translation invariant. It can be shown that g is ergodic if and 
only if a is irrational. For a proof and more examples, see [T9] . 

The following is one of the core theorems of ergodic theory. We will only use it to prove 
the corollary which follows, so it need not be read in detail. The proof can be found in any 
text on ergodic theory, e.g. [T9] . 



Theorem 4.10 (Birkhoff's ergodic theorem). Fix a probability measure fi and a fi-measure 
preserving transformation g. Then for any f G £ 1 (/i): 

• f(x)= lim^oo - X]fc=o f(9 k ( x )) exists ^-almost everywhere, 

• ffdfi = J fd/i, 

• f{9( x )) = fi x ) ^-almost everywhere, and 

• if g is ergodic then f(x) = J f dfi fx- almost everywhere. 

Corollary 4.11. Suppose // and v are probability measures such that v is absolutely contin- 
uous with respect to fi. If a transformation g preserves both /i and v and g is ergodic with 
respect to fi, then v = //. 
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Proof. Fix any measurable set A. Let / be the indicator function for A, i.e. the function 
equal to unity on A and zero elsewhere. Applying Birkhoff's ergodic theorem to / and fi 
yields f(x) = f-i(A) /^-almost everywhere. Since v is absolutely continuous with respect to /i, 
f(x) = jJi{A) //-almost everywhere also. If we now apply Birkhoff's ergodic theorem to v we 
get: 

v(A)= [fdu= ffdu= f fi(A)du = n(A). □ 



We now construct a family of extreme correlated equilibria. 

Proposition 4.12. Fix measures z/ 1; u 2 , v 3 , and z/ 4 G A*((0, 1]) and maps fi : (0, 1] — > (0, 1] 
such that — Vi o ff (interpreting subscripts mod A). The portion of the measure [l in 
the i th quadrant of I x I will be constructed in terms of fi and z/j. Define ji : (0, 1] — > I x I by 
ji(x) = (xj^x)), j 2 (x) = (-f 2 (x),x), h(x) = (-x,-f 3 (x)), and j 4 (x) = (f 4 (x),-x). Let 



-i 



= 1 Vi ° ]■ 

1 d\n\ is a correlated equilibrium 



If Assumption 2.1 is satisfied, supp|«;| C Cx x Cy, 



and t^-t 



l«l = EI 

then dfi — 

By assumption f^ o f 3 o f 2 o fx : (0, 1] — > (0, 1] is v\-measure preserving. If it is also 
ergodic with respect to v\, then fi is extreme. 

Proof. First we must show that [l is a correlated equilibrium. It is a finite measure by the 
assumption ^ G £ 1 (|k|) and xy ^ /i-a.e. by definition. Define g : I x I — > I x I &s follows. 



ji(x) if x > 0, y < 

j 2 (y) ifx>0,y>0 

j 3 (-x) ifx<0,y>0 

j 4 (-y) ifx<0,y<0 

arbitrary otherwise 



The function g is |fi;|-measure preserving. To see this fix any measurable set B C (0, 1] x (0, 1]. 
Let A = 3i\B). Then \k\{B) = \k\(A x (0, 1]) = Vx{A) by definition of |«|. But g^{B) = 
r 1 (Ax(0,l]) = Ax[-l,0),so 



\k\{ 9 -\B)) = \k\{A x [-1,0)) = Vi(jZ\A x [-1,0))) = Mf^iA)) = V\(A) = \k\(B). 

Therefore g is measure preserving for subsets of (0, 1] x (0, 1]. The arguments for the other 
quadrants are similar and since g maps each quadrant into a different quadrant, g is measure 
preserving on its entire domain. 

Define the signed measure k by dn = xyd/j = sign(a;) sign(y) d|/c|. We have seen that 
\k\(A x (0, 1]) = \k\(A x [-1, 0)), so k{A x (0, 1]) = -k(A x [-1, 0)). Since k(A x {0}) = 0, 
we have k(A x I) = 0, or using the terminology of Proposition |4.4 k x (A) 
argument implies k x (A) 



0. A similar 
by definition of 



if A C [-1,0). Clearly ^(0) 
is the zero measure. In the same way we can show that n y is the zero measure, so /i is a 



correlated equilibrium by Proposition 4.4 
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Now we will show via several steps that /x is extreme. Write fi = fix + /J2 where the fa 
are nonzero correlated equilibria. Since these are all positive measures, the fa are absolutely 
continuous with respect to fa Define d|«j| = \xy\ dfa and <i/«j = xydfa. 

Next we show that (7 is |ftj|-measure preserving. We will demonstrate this fact for B C 
(0,1] x (0,1]. As above, we define A = ]i\B). Then KK-B) = |^|(A x (0,1]) since 
(A x (0,1]) A B has measure zero and is absolutely continuous with respect to 
|«|. Furthermore, |Kj|(<7 _1 (.B)) = x [—1,0)). But fa is a correlated equilibrium so 

Ki(Ax (0,1]) = -k«(Ax [-1,0)). Hence I^Qr 1 ^)) = \ Ki \(A x [-1, 0)) = |/q|(A x (0, 1]) = 
Again, the proof is the same for B contained in other quadrants, so g is (-measure 
preserving. 

For the second-to-last step we prove that g is ergodic with respect to |ac|. Suppose 
B C I x I is such that \n\(g~ 1 (B) A B) = 0. Let Qj be the intersection of B with the 
z th quadrant. Then \K\(g-\Q i+l ) A £*) = 0, so H(cT 4 (<2i) A Qi) = 0. Let A = jT^Qi). 
Then |k| ((7~ 4 (Qi) A Q\) = V\{{fi /3 fi fi) _1 (A) A A) = 0. By assumption the map 
/ 4 o/ 3 o/ 2 o /j is ergodic, so z^(A) = or v x {A) = ^((0, 1]) = |/c|((0, 1] x (0, 1]). Therefore 
N(Qi) = u i{A) = or |k|((5i) = |/c|((0, 1] x (0,1]). In either case since g is |/t|-measure 
preserving we get H(Qj) = |/c|(Qi) for all i. Therefore |«|(-B) = or |/«|(-B) = \k\(I x 7), so 
g is ergodic with respect to \k\. 



Normalizing |ac| and to be probability measures, we can apply Corollary 4.11 to obtain 

Kj\(IxI) 

\k\(IxI) 



\Ki\ — tmjztw \ By definition the set on which \xy\ is zero has /i measure zero. Therefore 



1 H(/x/) 1 NC^jjO , 

^ - 1^1 d\Ki\ - N(/ x 7) Nl - x 7) ^ 



so ^* = \k\{IxI) ^ anc ^ A* i s extreme. 



□ 



Above we have constructed /i and g so that g maps the quadrants counter-clockwise - 
quadrant 1 to quadrant 2, etc. However, the same argument would go through if g mapped 
the quadrants clockwise. 



To view Proposition 4.5 as a special case of Proposition 4.12 , let each z/j be a uniform 
probability measure over a finite subset of (0,1]. The function g is defined by g(xi,yi) = 
(xi+i, yi+i) and the /j are defined to be compatible with this. The map / 4 o f 3 o f 2 o f 1 is a 
permutation on the support of u\, which is precisely the positive values of Xj. By construction 
this permutation consists of a single cycle, hence it is ergodic. 



Example 2.3 (cont'd). We can combine Example 4.9 and Proposition 4.12 to exhibit extreme 
points of the set of correlated equilibria for this game which are not finitely supported. Let 
< a < b < 1. Let z/j be Lebesgue measure on [a, b) for all i. Fix a such that is 
irrational. Define fi : [a, b) — > [a, b) by f(x) = (x — a + a mod (b — a)) + a. This is just an 



affinely scaled version of Example 4.9 so j\ is z/j-measure preserving and ergodic. Define f\ on 



(0, 1] \ [a, b) arbitrarily, because that is a set of measure zero. Let /2, ^3, fi : (0, 1] — > (0, 1] be 



the identity. These data satisfy all the assumptions of Proposition 4.12 . In particular, since 
< a < 6 < 1, xy is bounded away from zero on the support of \k\. Therefore r^r G £ 1 (|k|). 
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Figure 3: The support set of an extreme correlated equilibrium which is not finitely sup- 
ported. Extremality of this equilibrium depends sensitively on the choices of endpoints for 
the line segments. In this case there are segments connecting: (0.2,-0.2) to (0.8,-0.8); 

(-0.2,-0.2) to (-0.8,-0.8); (-0.2,0.2) to (-0.8,0.8); ( 0.2, 0.2 + to f 0.8 



and 0.8 



— 2 



to 



0.8,0.2+^ 



Since V{ is not finitely supported, /i is an extreme correlated equilibrium which is not finitely 
supported. The support of fi is shown in Figure [3] with parameters a = 0.2, b = 0.8, and 

a = 7E- 

Definition 4.13. Given a compact Hausdorff space K we say that a set of measures Ai C 
A* (K) is describable by moments if there exists an integer d, bounded Borel measurable 
maps g±, . . . , g& '■ K — > M, and a set M C ]R d such that a measure /i is in M. if and only if 
( f gidfi, . . . , J g d dfi) e M. 

The results of [12] show that the maximin and minimax strategy sets of a two-player zero- 
sum polynomial game can always be described by moments. Introducing a similar notion for 
n-tuples of measures, the set of Nash equilibria can always be described by moments in any 
polynomial game |21j . However, combining this example with the following proposition we 
see that the set of correlated equilibria of a polynomial game cannot in general be described 
by moments. 

This is important because the finite-dimensional representation in terms of moments is 
the primary tool for computing and characterizing Nash equilibria of polynomial games. 
One is therefore naturally drawn to try to find such a representation for the set of correlated 
equilibria. The example and this proposition show that no such representation exists in 
general. 
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Proposition 4.14. Let Ai C A*(K) be a set of measures describable by moments. Then all 
extreme points of Ai have finite support and this support is uniformly bounded by d, where 
d is the integer associated with the description of Ai by moments. 

Proof. Let g\, . . . y g& : K — > R be the maps describing Ai. Suppose there exists a measure 
fi G Ai which is extreme and supported on more than d points, so we can partition the 
domain of fi into d + 1 sets B\, ... , Bd+i of positive measure. For c = (ci, . . . , Cd+i) G R>o\ 
define /x c = Ylt=i C i/^I-B»- The map c i— >• /i c is injective. Define 



K — <^c G R^q 1 y ft ^/i c = J 9i dp for i = 1, . . . , d 

so(l,l,...,l)ei(r. Linearity of integration implies that the nonempty set K is the intersec- 
tion of an affine space of dimension at least one with the positive orthant. By Caratheodory's 
theorem (or equivalently the statement that a feasible linear program has a basic feasible 
solution), the extreme points of K each have at most d nonzero entries [2]. Thus (1, 1, . . . , 1) 
is not an extreme point of K, so we can write (1, 1, . . . , 1) = Ac + (1 — A)c' for < A < 1 
and (1, 1, . . . , 1) 7^ c, d G K. Therefore \i = = AyU c + (1 — A)/v is not extreme. □ 



5 Future work 

These results leave several open questions. If we define a moment map to be any map of 
the form n i— > (J fidir, j fedir, . . . , J fk d%) for bounded Borel measurable fi, then we have 
shown that the set of correlated equilibria is not the inverse image of any set under any 
moment map. On the other hand, since moment maps are linear and weak* continuous, we 
know that the image of the set of correlated equilibria under any moment map is convex and 
compact. 

Supposing the utilities and the fi are polynomials, is there anything more we can say 
about this image? In particular, is it semialgebraic (i.e., describable in terms of finitely 
many polynomial inequalities)? If so, can we compute these inequalities or a solution thereof 
efficiently for given utilities? A sequence of easily computed outer bounds to this image is 
presented in [22]. Can we compute nonempty inner bounds? 
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